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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to comnnunlcation(s) filed on 06 May 2005 . 
2a)S This action is FINAL. 2b)n This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn frqm consideration. 

5) 0 Claim(s) 2,3 and 15-18 is/are allowed. 

6) ^ Claim(s) 1.4-9 and 11-14 is/are rejected. 

7) 13 Claim(s) 10 is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Appljcation Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction Is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or fomi PTO-152. 

Priority under 35 U.S.C. § 119 

12)n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Response to Amendment 

1 . Applicant's amendment filed on May 6, 2005 has been entered and made of 
record. 

In response to applicant's submission of Replacement Drawings are accepted. 

Response to Arguments 

2. Applicant's arguments filed on May 6, 2005 have been fully considered but they 
are not persuasive. 

Claimsl 5-20 are new. 

Claims 1-20 are pending in the application. 

Allowable Subject Matter 

3. The following is an examiner's statement of reasons for allowance: 

Claim 2 and 3 is allowed as a result of applicant's amendment by including the 
allowable subject matter as set fourth in paragraph 8 of the last office action mailed on 
2/8/05. 

For Independent claims 15 and 18, the prior art on record, fails to teach or fairly 
suggest, singly or in combination, a method for deriving a three-dimensional model of 
scene said method comprising, among other things, wherein each of said three- 
dimensional panoramic images is derived from a respective set of range images, each 
said set having a different nodal point. 

Claims 15-20 are allowed. 
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REMARKS 

4. Applicant's arguments regarding clainn 1, have been fully considered and with 
respect to the art rejection, the examiner has carefully considered applicant's argument, 
but firmly believes the cited reference to reasonably and properly meet the claimed 
limitation. The examiner does not agree with the remarks. On page 10 applicants' 
argues, " the invention can be used to create full view panoramic mosaics from image 
sequence. Unlike current panoramic stitching methods, which usually require pure 
horizontal camera panning ... panoramic mosaics". This limitation is not recited in 
claims and claim language does not exclude. 

On page 10 applicant's argues " By taking as many images as needed image 
mosaics can be constructed which cover as large a field of view the virtual 
environment can be viewed or explored using standard 3D graphics viewers and 
hardware without requiring special - purpose players". This limitation is not recited in 
the claims. 

On page 1 1 applicants' argues, "the specification discloses how to solve the 
problem of loss of detail or image ghosting is solved by computing local motion 
estimates (block-based optical low) between pairs of overlapping images, and using 
these estimates to warp each input image so as to reduce the misregistration. This is 
less ambitious than actually recovering a perspective depth value for each pixel, but has 
the advantage of being able to simultaneously model other effects such as radial lens 
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distortions and small movements in the image." However, applicant is reminded that the 
claim language is given its broadest reasonable interpretation and applicant cannot rely 
upon the features from the specification brought out in the claims because the 
specification is not the measure of the invention but claims are. 

On page 10 applicant's argues about Szeliski does not teach or suggest step (a) 
of claim 1 . With respect to the art rejection, the examiner has carefully considered 
applicant's argument, but fimrily believes the cited reference to reasonably and properly 
meets the claimed limitation. The examiner does not agree with the remarks that 
Szeliski cannot be said to suggest, "generating a plurality of three-dimensional 
panoramic images of a scene" (column 4, lines 45-65). Szeliski is relied upon to provide 
these features as stated in the rejection. However, applicant is reminded that the claim 
language is given its broadest reasonable interpretation. Therefore, Szeliski does 
disclose this limitation. 

On page 1 1 applicants' argues about the panoramic mosaics of Szeliski lack 
depth information. Applicant arguments are not valid because examiner has brought in 
secondary reference to support the claim limitation. Therefore, Huber constructs a 3D 
model from plurality of range images using surface matching algorithm. 

On page 10 applicant's argues about Szeliski does not teach or suggest step (b) 
of claim 1 . With respect to the art rejection, the examiner has carefully considered 
applicant's argument, but firmly believes the cited reference to reasonably and properly 
meets the claimed limitation. The examiner does not agree with the remarks that 
Szeliski cannot be said to suggest " b) determining transformation (column 27, lines 14- 
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27) that align the plurality of three- dimensional panoramic images " (figures 4-5, also 
see column 13 line 25 to column 14 line 45, column 20, lines 10- 52). Szeliski is relied 
upon to provide these features as stated in the rejection. 

On page 12 applicant's argues about Szeliski does not teach or suggest step (c) 
of claim 1 . With respect to the art rejection, the examiner has carefully considered 
applicant's argument, but firmly believes the cited reference to reasonably and properly 
meets the claimed limitation. The examiner does not agree with the remarks that 
Szeliski cannot be said to suggest c) integrating spatial information from the plurality of 
three- dimensional panoramic images to form a spatial three-dimensional model 
(column 27, lines 14 - 27) scene (figures 1 and 2B; environment/ texture map memory 
270 stores a 3-D model, column 31, lines 45- 49). Szeliski is relied upon to provide 
these features as stated in the rejection. Therefore, Szeliski does disclose this limitation. 

On page 13 applicant's argues about Szeliski does not teach or suggest step (d) 
of claim 1 . With respect to the art rejection, the examiner has carefully considered 
applicant's argument, but firmly believes the cited reference to reasonably and properly 
meets the claimed limitation. The examiner does not agree with the remarks that 
Szeliski cannot be said to suggest (d) integrating intensity and texture information (fig 2) 
from the plurality of three- dimensional panoramic images (fig 3, column 19, line 65 
through column 20, line 9, 1 1-27) onto the spatial three-dimensional model (column 31, 
lines 14- 42) to form a three-dimensional model of the scene (fig 2B, element 270, 
column 31 , line 53 through column 32, line 42) containing both spatial and intensity 
information (figure 11-12). Szeliski is relied upon to provide these features as stated in 
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the rejection. However, applicant is reminded that the claim language is given its 
broadest reasonable interpretation. Therefore, Szeliski does disclose this limitation. 

Claim Rejections '35 use §103 
5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject nnatter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
This application currently names joint inventors. In considering patentability of 

the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 

the various claims was commonly owned at the time any inventions covered therein 

were made absent any evidence to the contrary. Applicant is advised of the obligation 

under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 

not commonly owned at the time a later invention was made in order for the examiner to 

consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 

prior art under 35 U.S.C. 103(a). 

5. Claims 1, 4.7-9 and 13-14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over a combination of Sezliski et al. (US 6,044,181) and Huber (automatic 
3D modeling using range images obtained from unknown viewpoints—IDSj. 
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With regard to claim 1, which is representative of claim 9, Szeliski discloses a 
method for deriving a three-dimensional model of a scene from a plurality of images of 
the scene (figures 1-3), said method comprising: 

(a) generating a plurality of three-dimensional panoramic images of a 
scene (figures 2-3; column 4, lines 45-65); 

(b) determining transformations (column 27, lines 14- 27) that align the 
plurality of three-dimensional panoramic images (figures 4-5, also see column 13 line 25 
to colunrin 14 line 45, column 20, lines 10- 52); 

(c) integrating spatial infomnation from the plurality of three-dimensional 
panoramic images to form a spatial three-dimensional model (column 27, lines 14-27) 
of the scene (figures 1 and 2B; environment/ texture map memory 270 stores a 3-D 
model (column 31 , lines 45- 49); 

(d) integrating intensity (column 17, lines 12-25) and texture information 
(figure 2B) from the plurality of three-dimensional panoramic (column 19, line 65 
through column 20, line 9) images (figure 3) onto the spatial three-dimensional model to. 
form a three-dimensional (column 31, lines 14- 42) model of the scene (270 in figure 2B, 
column 31 , line 53 through column 32 line 42) containing both spatial and intensity 
information (figures 11-12). 

Szeliski is silent about range images as claimed. Huber, in the same field of 
endeavor of automatic 3D modeling using range images obtained from unknown 
viewpoints (title) and same problem solving field of range image information disclose the 
use of range images. In particular, Huber constructs a 3D model from a plurality of 
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range images using a surface matching algorithm. It would have been obvious to a 
person of ordinary skill in the art at the time of the invention was made to modify 
Szeliski by using range images in creating 3D panoramic image model as taught by 
Huber because such a modification will allow a system to create an automatically 3D 
model of a scene from a set of range images obtained from unknown viewpoints as 
mentioned Huber in the title and also at least in the abstract. 

As to claim 4, Szeliski discloses the method wherein step (b) further comprises: 

(a) determing one or more pairs of three-dimensional panoramic images 
that contains some common scene infonnation (figures 1, 2B, 3); 

(b) determining the transformations that align each pair of three 
dimensional panoramic images that contain some common scene information (figure 1. 
2B); 

(c) determining global inconsistencies In the transformations found in step 
(b) (column 20, lines 34-63). 

As to claims 7 and 13, Sezliski discloses the method wherein the three- 
dimensional panoramic image is a color image (column 28, lines 26-50). 

As torn claims 8 and 14, Huber discloses the method wherein one or more range 
images are juxtaposed between a pair of three-dimensional panoramic images (figure 
2). 

6. Claims 5 and 1 1 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
the combination of Sezliski et al. (US 6,044,181) and Huber (automatic 3D modeling 
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using range images obtained from unl^nown viewpoints— IDS) as applied to claim 1 
above, and further in view of Nayar et al. (US 4,912,336). 

The arguments as to the relevance of the aforesaid combination as 
applied above are incorporated herein. The aforesaid combination does not disclose 
Lambertian reflectance model. Nayar provides this model as shown in figure 2A and 
column 5 line 65 to column 6 line 20. It would have been obvious to one of ordinary skill 
in the art at the time of the invention was made to modify SzeliskI and Huber's 3D 
panoramic model generating system by adding a Lambertian reflectance model as 
taught by Nayar because such a modification would have allowed for a system to 
determine the parameters of the reflectance model at each surface point as shown by 
Nayar at column 2, lines 1 3-1 5). 

6. Claims 6 and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
a combination of SzeliskI et al. (US 6,044,181) and Daniel F. Huber (Automatic 3D 
Modeling Using Range Images obtained From Unknown Viewpoints— IDS) as applied 
to claims 1 , 4, 7-9 and 13-14 above and further in view of applicant's admitted prior art 
in the specification at page 10. lines 4-14. 

As to claims 6 and 12, Szeliski discloses the method, wherein the step (d) of 
integrating the intensity and texture information from the plurality of three-dimensional 
panoramic images (figure 2) but neither Szeliski nor Huber discloses a reflectance 
model that depends on the viewpoint of the observer. However, applicant's admitted 
prior art clearly shows the use of a reflectance model as mentioned at column 10, lines 
4-14. It would have been obvious to use a reflectance model in the method of Szeliski 
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as modified by Huber as taught by the admitted prior art. Doing so will yield a more 
accurate rendering that incorporates the differences in surface reflectance as a function 
of the viewing angle. 

Allowable Subject Matter 

7. Claim 10 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any inten/ening claims. 

8. Accordingly. THIS ACTION IS MADE FINAL. See MPEP 706.07(a). Applicant 
is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event will the statutory period for reply expire later than SIX 
MONTHS from the date of this final action. 
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Contact Information 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sheela C Chawan whose telephone number is. 571- 
272-7446.The examiner can normally be reached on Monday - Friday 7,30 - 4.00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Infomnation regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more infomiation about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

Sheela Chawan 
Patent Examiner 
Group Art Unit 2625 
August 8, 2005 


